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ESTIMATION FOR ALMOST PERIODIC PROCESSES 
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Processes with almost periodic covariance functions have spectral 
mass on lines parallel to the diagonal in the two-dimensional spectral 
plane. Methods have been given for estimation of spectral mass on the 
lines of spectral concentration if the locations of the lines are known. 
Here methods for estimating the intercepts of the lines of spectral 
concentration in the Gaussian case are given under appropriate con- 
ditions. The methods determine rates of convergence sufficiently fast 
as the sample size n — » oo so that the spectral estimation on the esti- 
mated lines can then proceed effectively. This task involves bounding 
the maximum of an interesting class of non-Gaussian possibly non- 
stationary processes. 

1. Introduction. The main objective of this paper is to present a con- 
structive method to determine the lines of support of spectra or, equivalently, 
the frequencies or periods of Gaussian harmonizable processes with almost 
periodic covariances under appropriate conditions. The processes consid- 
ered provide an interesting set of random processes that are generally not 
transient and not stationary and for which Fourier methods of analysis are 
helpful and meaningful. The lines of support of spectra are lines parallel to 
the diagonal in the two-dimensional spectral plane. A number of papers have 
discussed spectral estimation in this context but they all assume knowledge 
of the lines of support of the spectra. The novelty of this paper is in the 
presentation of constructive methods for estimating the lines of support un- 
der appropriate conditions with a rate of convergence good enough to imply 
bias and covariance of spectral estimation using the estimated support lines 
with the same asymptotic properties as if the actual lines of support were 
known precisely. 
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Almost periodically correlated processes have been considered by Alek- 
seev [1], Hurd [10], Gardner [5], Gladyshev [9], Gerr and Allen [7, 8], Dan- 
dawate and Giannakis [3], Leskow and Weron [14] and Lii and Rosenblatt 
[15]. Extensive application of periodically correlated (or cyclostationary) 
processes is noted in [6] and references therein. For the sake of simplic- 
ity and economy, we limit ourselves to a discrete time parameter process 
{X n , n = . . . , — 1, 0, 1, . . .} with mean EX n = 0. The process {^ n } with co- 
variance function r n ^ m = EX n X m is said to be periodically correlated with 
period T if 

(1-1) r n+ T,m+T = r n: m- 

We shall call {AT„} a process with almost periodically correlated covariance 
function if for each (s,t), there are functions otj(s,r) and real Xj(s,r) with 
J2j \ a j( s > T )\ < 00 such that, for each s,r, r s+t ,r+t = J2j a j ( s > r)e* Aj ^ ,T ^ with 
the convergence uniform in s,t. We shall call a sequence {a(n)} almost 
periodic if there are sequences {aj}, real {j3j} such that a(n) = J2j a j e 
with J2j \ a j I < 00 • 

The process {A n } is harmonizable (in the sense of Loeve) if it has a 
Fourier representation in mean square, 

(1.2) X n = f W e mX dZ(\), 

with the random spectral function having a covariance 

cov(Z(A),ZGu)) = F(A,^), 

(1-3) 

r(n,m)= T T e inX - im " dF(X, n), T [* \dF(X,fi)\ < oo 

J —TV J —7T J —TV J —TV 

[16, 20]. Almost periodicity of the covariance function of a harmonizable 
process implies that F has mass (complex) on at most a countable number 
of diagonal lines 

A = n + b, b = b j: J = - - - ,—1,0, 1, — 

If the process is real- valued (as we assume to be the case) and A = \i + b is 
a line of spectral support, then so is A = \i — b. 

If the spectra on the lines of support of F are absolutely continuous with 
respect to Lebesgue measure with spectral densities /&(//), the assumption 
that {Xn} is real- valued implies that 



(1.4) 



Mfi) = f-bfa + b) = /- 6 (-/i) = M-fi - b). 
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Also, 

(1-5) \Mv)\ 2 <fo(v)fo(v + b). 

Let us call C the collection of 6's corresponding to lines with nontrivial 
spectral support. In [3], the spectra /&(//) are estimated under appropriate 
conditions assuming that the elements of C are known. 

We consider estimation of b £ C under the assumption that £ is finite and 
Jl w fb(X) dX^O and see whether the final estimates of fb(^) based on these 
estimated elements of C produce an effect. 

At this point, a few additional remarks are made on related papers. Tian 
[18] considers time domain estimation of the integral period of a sequence 
with periodically correlated covariance function observed at integral time. 
Hurd and Gerr [11] and Gerr and Allen [7] use the coherence between spec- 
tral components to test for a specific period of a periodically correlated 
covariance function against the null hypothesis of stationarity. They used 
the integrated coherence along diagonal spectral support lines. Lund et al. 
[17] use an averaged squared coherence statistic to test existence of periodic 
correlation against stationarity. The case of a continuous parameter process 
{X(-)} is considered in [4] and [12]. 

Before the main discussion we describe a simple but interesting class of 
processes with almost periodic covariance function. Additional examples can 
be found in [6] and [20]. Set 

m 

(1.6) X n = £ cos{ Wj n)Y^} Tj , < Wj < tt, 

where the processes {Yn } are jointly stationary with means zero and the 
cross-spectral density of Y^\-) and Y^ k \-) is fjk(')- This commonly used 

model in communication is a mixture of amplitude modulated (AM) sig- 

(i) 

nals with carrier frequency Wj modulated by the signal Yn with possible 

relative delay tj. If the stationary processes {Y^} have the spectral repre- 
sentation 

(1.7) y B W= f j^dZjW, 

J — W 

the process {X n } is given by X n = J^ n e tnX dZ(X) with 

(1.8) dZ( v ) = i^Je-^^) dZjirj - Wj ) + e~ iT ^ +w ^ dZj{r) + Wj )\. 

j 

Since the vector of processes {Zj(X)} has orthogonal increments, the spectral 
mass of the process {A^n} is given by 
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(1.9) EdZ(X)dZ(p) 



3,k ) 
where the first sum is over (j, fc) such that 
A — \x = Wj ± Wfc = rj 7^ 

and the second primed sum over (j, A;) such that 



A 



-w 



0, 



j 

otherwise. 



± it;* = 77 7^ 0, 



The estimation method proposed in this paper can be used to estimate the 
frequencies Wj, the spectral densities of the K^'s and the Tj's under certain 
assumptions. We note that the remarks for almost periodic sequences imply 
that a moving average process 



Xn 



m 

3=0 



n-31 



E£ n = 0, 



with the sequences (in n) {aj(n)},j = 0, 1, . . . ,m, almost periodic and the 
process a white noise process, is a process with almost periodic covari- 
ance function. That is also true of the autoregressive scheme X n — a n X n -i = 
if {a n = J2j aje ll3jn } is an almost periodic sequence with J2j \ a j\ < 1 ( see 
also [13]). 

We note that in meteorological, astronomical and engineering applications 
questions of periodic or almost periodic covariances are often of interest (see 
[17] and [20] for examples). The organization of the rest of the paper is as 
follows. In the next section notation and assumptions are given. The main 
result on the estimation of intercepts of almost periodic spectral support 
lines using an integrated periodogram [see (2.6)] is presented after Corollary 
4. The estimation method is based on Theorem 4. Theorems 5 and 7 show 
that the asymptotic bias and variance of spectral estimates on the estimated 
lines are the same as those when the support lines are known. Section 3 con- 
tains a simulated example to demonstrate the effectiveness of the methods. 
The Appendix contains most of the proofs of the results. 

2. Results. In this section we introduce notation and assumptions, and 
state the main results. The principal thread of the derivation will be in- 
dicated. However, the detailed proofs of the results will be given in the 
Appendix. By 



(2.1) 



W = Zmod27r, 
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it is to be understood that — ir < W < ir and if {u} is the integer I with 
-1/2 < u-£< 1/2, then (Zmod2vr) = Z - {Z / {2tt)}2tt . Consider the finite 
Fourier transform 

n/2 

(2.2) F n (X)= ]T X tz~ UX 

t=-n/2 

and the periodogram 

1 



(2-3) Z„(A,/i) = ^—— F n (\)F n (jj.). 

Here we assume that n is even and t is summed over integers. Let K(rj) 
be a continuous nonnegative symmetric weight function with finite support 
and such that /f^ K(rj) drj = 1. Set K n {rj) = b~ l K{b~ l if) with b n [ and 
nb n — > oo as n — > oo . A plausible estimate of / w (77) is given by 

J n (/x + io, /i)K n (/x - rj) dfj,. 

Assumption 1. = 0, is a harmonizable Gaussian sequence 

with almost periodic covariance function. The set C of b's with nontrivial 
support is finite, and the /&(//),& G £, are continuously differentiable on the 
curve A = (b + fi) mod27r on the torus of points (A, /i) G [— vr,7r] 2 , with (— 7r,/j) 
identified with (7r,/i) and (A, — 7r) with (A,7r). 

Now 



= £X n A m = ^ r e i <» + Ve- im »f b (n)d l i = J2e inb Cb(n 
beC J ~' K bee 

(2-5) 

c 6 (n) := / e™ x h(X)d\. 



Assumption 2. There is a function c(j) > on the integers with |c&(j)| < 
c(j) for all b G C and c(j) < 00. Unless otherwise noted, the summation 
over j is on the integers. 

Assumption 3. The fj,(n),beC, are such that JZ n /&(//) d/j.^0. 

In general, it is possible that there may be spectral lines b with /&(//) d/J : 
(see [11], page 347). In that case let C consist only of those b with 
J-tt fb(n) d[i 7^ and our arguments will be valid for the restricted C The 
local extrema of the absolute value of the integrated periodogram 



(2.6) U n {b):= I n (b + n, (i )dii=—- X l 



r2 e -ibk 
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will be used to estimate the b G C under Assumption 3. The fact that (2.6) is a 
periodogram computed for X% is due to the uniform weighting in integrating 
I n (b + ^,/Lt). With nonuniform weighting, one would get more complicated 
quadratic forms that might be useful if fb(l^) d/i = 0. 

Theorems 1 and 2 give estimates for EI n (X, /j,) and cov(I n (fi + w, «),/„(// + 
w',fi')) and it is crucial that they are given with error terms uniform in the 
w's and /i's. Let 

(2.7) y(b,w) = (b- w)mod27r. 

It will also be convenient to make use of the following expressions for b, b' 6 C 
when A = fi + w, X' = // + w'\ 

y (l) = (X' + b- A)mod2vr, 

y (2) = (-n' + fi + b')mod2ir, 

y (3) = (-// + b - A) mod 2vr, 

y (4) = (A' + /i + 6 / )mod27r. 



(2.8 



Also let 



(2.9) sinc(y) = sin (j-^-v) / ' 
Theorem 1. IfX = fi + w, 

(2.10) mux,,) = £ AM 8 'l ( "^ 1 ; ) / gfl': )) + o^'° g " 



((n + l)/2)!,(6,w) V n 

with uniformity in w,[i when Assumption 1 is satisfied. 

Theorem 2. Assumption 1 and X = fj, + w, X' = fi' + w' imply that 
cov(/ n (/i + n),I n {v! + w', //')) 

' log n N 



^/ 6 (// + n/)sinc(n(l)) + 



n 



(2.11) x | £ fv{-ll) smc(y(2)) + o(^) 



+ fe/&(V)sinc(y(3)) + 



logn 



n 



EA'(^ + ^)sinc(y(4)) + 



logn 



n 
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Theorem 2 almost immediately implies Corollary 1. 

Corollary 1. Assumption 1 implies that 
cov(f w (r))J w >(r)')) 

/TT r 71 
I f h {y! + w')f b >(-n') sinc(y(l)) sinc(y(2))i^ n (^ - rf) 
0,0- ti. " 7r " / - 7r 

x K n {fj! — rf) dfx dfj, 

+ E r r A(V)AV W)sinc(y(3))sinc(y(4)) 
b,b , eC J ~ 7rJ ~ 7T 

x K n (n - r))K n (fi' - rf) d\x d/j/ 

+ /lognX 



(2.12) 



n 



Further, 

(2.13) cov(fM,L'(v')) = O (J^ + O (^p) . 

Bounds on sup w \U n (w) — EU n {w)\ will be obtained by making use of 
Theorem 3. Let 

(2.14) V n (w) = (n + l){U n (w) - EU n (w)}. 

Cumulants of V n (w) will be estimated and an argument akin to Brillinger 
[2] will be used to gauge sup^ | V n (w)\. 

Theorem 3. Let {X^} be a Gaussian sequence with almost periodic 
covariance function satisfying Assumptions 1 and 2. Then 



(2.15) limsupsup|K(w)|/(nlogn) 1/2 < 2 5/2 g ^c(j) , 

n— kx) w \ ' J 

with probability 1 where q is the number of elements in C 

The arguments used to prove Theorem 3 will imply the following two 
corollaries. 

Corollary 2. Let c > 0. Then under the conditions of Theorem 2>, 
there is a constant C > such that 



(2.16) P 



sup\U n (w) - EU n (w)\ > c 



<exp[-GV /2 (logn) 1/2 ]. 
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Corollary 3. If c> is fixed but large, 



(2.17) P 



sup 

w 



\U n (w) - EU n {w)\ > 2cn" 1/2 (logn) 1/2 



< exp< — -logn >. 



An additional corollary indicates that sup^ |V^(u;)| diverges as n — > oo. 

Corollary 4. Assume that fo(-) is not identically zero. Then under 
Assumptions 1-3, there are a countable number of distinct Wi in the neigh- 
borhood of zero such that, for each finite subcollection C, 

(2.18) -==V n {wi), Wi eC 

\Jn + 1 

are jointly asymptotically normal and independent with variances each greater 
than a constant £ > 0. Further, 



(2.19) 

in probability as n — > oo. 



1 



sup\V n (w) 



We now describe the estimation procedure. Fix a given level 5 > 0. Con- 
sider the number of b € C with 



(2.20) 



MfJ>) dfj, 



>S. 



The following procedure will yield estimates w n (b) of b G £ satisfying (2.20) 
which are such that n\w n (b) — b\ < a n = n" 1 / 5 except for a set of probability 
less than 0(logn/n 2 ) if n is sufficiently large {i.e., as n— ►oo). Divide the 
range (it can be taken as —ir < w < ir) of w into blocks of length c(n)/n 
[with lim n _> oo c(n) = oo and c[n) = o(n) as n —> oo, e.g., c(n) = logn or 
c(n) = re 1 / 2 ] . . . and compute the global maximum of \ + w, fi) dfi\ in 

each block. Throw out the blocks with maxima that fall less than or equal 

are two maxima located within c{n)/n of 
each other, throw out the block with the smaller maximum. As n—> oo, the 
locations w n (b) at which the remaining maxima occur satisfying (2.20) are 
such that n\w n (b) — b\ <a n = re~ 1//5 except for a set of probability less than 
0(logre/n 2 ) asn— >oo. 

We sketch the argument for this. Using (2.10), it is seen that 



EU n {w) 



^ sin((re+l)/2(fr 

bee 



w)) 



(n + 1)/2(6 - w) 



The maximum of 



(2.21) 



sin((n + l)/2)(6- w) 



((n + l)/2)(b-w) 



f b (fj,)df, + 



fb(fJ>)dn 



logn 



n 
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is clearly at w = b given Assumption 1. The contribution from the other 
terms in \EU n (w)\ with b' 7^ b,b' G C is asymptotically O(-) in a neighbor- 
hood {w:\b- w\ < mmv^b.b'^c \\b' — b\} of b G C. First note that, uniformly 
in w, 



(2.22) 



|CA n H|-|f?C/„H||<Ci 



log re 



n 



1/2 



except on a set of probability less than 0(n~ 2 ) by Corollary 3. We shall 
show that the local maximum w n (b) converging to b must be such that 
\n(w n (b) — 6)| < re -1 / 5 except for a set of probability less than 0(logn/n 2 ). 
First note that 

I n (fi + w,n)dfi 



fb(fJ>) dfi 



sm((n + l)/2)(b-w) 



{{n + l)/2){b-w) 



log re 



11 



1/2 



in a fixed nontrivial interval I(b,e) = {w: \b — w\ < e},0 < e < ^, contain- 
ing b but excluding the other points b' G C except for a set of probability 
0(logn/n 2 ). The value of \ J I n (fi + w, fi) d/i\ at w = b is at least 



®:= 



C 2 



logn 



?? 



1/2 



However, the value outside \n(w — b)\ < n 1 / 5 , but inside I(b,e), of | / I n (^ + 
w,fj,) dfi\, is at most 



fb{n)dfj, 



sinn 



-1/5 



n 



-1/5 



+ C 2 



log re 



n 



1/2 



and since 



sinn 



-1/5 



J? 



-1/5 



1 



n 



-2/5 



3! 



+ 0(n 



-4/5> 



0is larger than (2)for n sufficiently large. Consequently, we have the following 
theorem. 



Theorem 4. Let 5 > and Assumptions 1-3 ZioZd. There are then esti- 
mates w n (b) of values b G C with \ /&(//) du| > <5 suc/i i/iai n\w n (b) — b\ < 
n" 1 / 5 except for a set of probability 0(logn/re 2 ) if n is sufficiently large. 

A similar but more detailed argument yields a better estimate for \w n (b) — 

b\. 
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Corollary 5. Under the assumptions of Theorem 4, (n+l)(6— w n (b)) = 
O^-V^logn) 1 /*) and [{n + 1)(6 - w n {b))} 2 = 0{sup w \U n (w) - EU n (w)\) 
except for a set of probability 0(logn/n 2 ). 

The following results show that if one uses these estimates w n (b) of b to 
estimate the spectral densities at b, the bias and variance are asymptotically 
as good as for the estimates with b E C given. 

Theorem 5. If \ /&(a0 dfj,\ >5 and Assumptions 1-3 hold, 

Efw n {b){v) 



E 



sm((n + l)/2)y(b,w n (b)) 
((n + l)/2)y(6,w n (6)) 
and the right-hand side is 
bl 



f b (n)K n (fj,-r)) dpt 



+ o 



logn 



n 



fb(p) + ffb(p) J ^ * 2 K(x) dx + 0(b 2 n ) + CKn-^log n ) 1 ' 2 ) + 0^— 

Corollaries 2 and 3 are used in the derivation of this result. 
Theorem 6. Let 5 > and Assumptions 1-3 hold. If 



fb(p)dn >S, 

J — IT 

for b,b' 6 C, then 

cov(f Wn (b)(v), L n (b')(v')) = O 
as n — > oo, b n [ 0, nb n — > oo. 



fb' (p) dp 



1 

nb n 



+ 



>5 



logn 



n 



Theorem 7. If \ h{p) dfi\ > 5 and Assumptions 1-3 hold with / (-) 
not identically zero, setting j(x) = 1 if x = mod 2-7T and otherwise, then 



var(/ Wn{6) (??)) 
2tt 



fo(v + b)fo(-v) 



+ Y J l{-2r 1 + b'-b)\f v {-r ] )f 

b'eC 



/oo 
if 2 (a;)d2; + 



logn 



A comparison of Theorems 5 and 7 with Proposition 5.1 and Corollary 
5.1 in [15] indicate that the bias and variance of the spectral estimates using 
estimates w n (b) of b G C have the same behavior asymptotically as spectral 
estimates assuming complete knowledge of b £ C. 
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3. Examples. In this section we give a simulated example. Consider the 
amplitude modulated (AM) signal model in communication 

(3.1) y(t) =cos(uj x t)Z 1 {t) + cos{u 2 t)Z 2 (t), EZi(t)=0, z = 1,2, 

where Z\{t) and Z 2 (t) are independent stationary processes with spectral 
densities /i(A) and /2(A) at frequency A, respectively. Then {y(t)} is harmo- 
nizable with almost periodic covariance function and spectral support lines 
on A = fj,, A = fi±2ui and A = /i±2u;2- We consider (3.1) with uj\ = j^=, us 2 = 
and Zi(t) =e t + 0.5e t -i, Z 2 (t) =i] t + 0.3rj t -i, where e t and rj t are inde- 
pendent standard Gaussian sequences. In this case, y(t) = J e ttx dZ y {\) with 
dZy(X) = i[(fei(A=Fwi) + dz 2 (\^uj 2 )} andZi(t) = Je ltx dzi(X) fori = 1,2. 
Therefore, the spectrum of {y(t)} is given by 

E[dZy(\)dZjii)] 

= i{(/i(At^i) + /2(A T ^ 2 ))V 

+ /i(A T wi)5a, m ±2^i + /2(A T u 2 )5\ tlJi ± 2uJ2 }. 

Given a realization of {y(t)}, we are interested in estimating u\ and uj 2 and 
the spectral densities on the lines given by A = fi ± 2ui and A = fj, =t 2cl>2. 

As an example, a simulated realization of {y(t)} for t = —512 to 512 is 
generated. A stretch of {Zi(t)} and {y(t)} are given in Figure 1. The peri- 
odogram of {y(t)} is given in Figure 1 also. There is no apparent information 
on uo\ and u 2 in the periodogram. The U n (b) statistic of {y{t)} is computed 
for a range of 6's. For b = b j = j^ E with j = 1, 2, . . . , 512, U n (b) and E(U n {b)) 
are given in Figure 2(a)-(f). 

Two spikes are apparent at frequencies near 2u>i and 2uj 2 . The largest two 
6's in \U n {b)\ occurred at fa = 0.45406 and Cj 2 = 0.73938. When the U n (b) 
are calculated at a more refined grid with b = bj = Jj^f, j = 1, 2, . . . , 2048, 
the largest \U n (b)\ occurred at Cj\ = 0.45329 and uj 2 = 0.74015. These latter 
values are closer to the true values uj\ = 0.45345 and oj 2 = 0.74048. These 
are plotted in Figure 2(g) and (h). The scale in figures (g) and (h) differs 
from that in figures (e) and (f) since there is leakage at frequencies very 
close to zero because the computation at (e) and (f) is at a more refined set 
of frequencies. The zero frequency corresponds to the diagonal line which 
is the support line of the "power spectrum" of the process. Therefore, the 
value of |J7 n (0)| is relatively large. It is this mass that leaks. 

A large value of \U n (b)\ suggests that A = \x + b is a spectral support line. 
Figure 3 gives the plots of spectral estimates on these estimated support 
lines. The dashed line gives the theoretical spectral density if the intercept b 
is known exactly. In these cases the imaginary part is zero. For the diagonal 
line X = n, there is no need to estimate the intercept. The estimated spectral 
densities on each line when the intercepts are exactly known are given by 
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the solid lines. The dotted lines are spectral estimates when the intercepts 
are estimated with a crude search, while the dash-dot lines are spectral 
estimates when the intercepts are estimated by the more refined search. 

When the sample size of {y(i)} is increased to t = —2048 to 2048, we have 
more resolution as expected. 

These graphs show that intercepts of these support lines or frequencies or 
"periods" of the covariance function can be effectively estimated using U n (b). 
These intercepts can be estimated sufficiently accurately to give accurate 
estimates of the spectral densities on the spectral support lines. 

APPENDIX 

For the proofs, the following will be useful. Let 

sin((n + l)/2)x ~ sin((n + l)/2)x 

(A.l) V n {x) - gin(a;/2) , D n {x)- — . 

Lemma A.l. If \y\ < tt } 

/TT f"K 
D n {x + y)D n {-x)dx = / D n (x + y)D n (-x) dx + O(logn), 
-7T J — 7T 

with O uniform in y. 

The argument given in [15] for Lemma 7.2 holds for this case. 
Proof of Theorem 1. Given X t ,t = — n/2, . . . ,n/2, then 

\ PIT 

EI n (X,fi) = — — — V / D n (v + b- fi-w)D n (-v + n)f b (v)dv 
(A.3) 

bec 

with Gi = 27r (n+i) I-7T D n( v - V + y(b, w))D n {-v + fJ,)fb(v) dv. The continu- 
ous differentiability of /{> implies that 

(A.4) fb(v) = Mri + (v-ti)f b (v*), 

with v* between \i and v. Then G\ = G2 + G3 with [denoting y(b, w) by y] 

° 2 = n i "Tn / D ^ v ~ ^ + v) D n(-v + n)(v - fi)fi(v*) dv, 
2ir(n + 1) J- n 



1 f n 
G3 = ——, — —rrfbW / D n {x + y)D n (-x)dx. 
27r(n + l) J-,r 
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Fig. 2. U n (b) statistics of{y(t)}. Frequencies are computed for increments o/27r/1025 
for (a)-(f) and for increments of 27r/4097 for (g)-(h). 
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<n,.j 0.74CHB, ttillmBlWl » ? » 0.73338. 7«1 5 




Fig. 3. Spectral density estimates at different intercepts; the theoretical part is given 
by the dashed - - - line, estimated part with intercept known is given by the solid line; 
estimated part with intercept estimated at lower resolution is given by the dotted ■ ■ ■ line; 
estimated part with intercept estimated at higher resolution is given by dash-dot - ■ - line. 
Both real and imaginary parts are given. Theoretical imaginary parts are all zero and they 
are not plotted. 
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It is clear that |G 2 | = O(^). Also, 

G3 = M(i) . x1 . A, (a; + y)D n {-x) dx + ol -2- 
using Lemma A.l. To simplify the notation, we define 



J2 f )f(x)dx = Y / f f(x)dx. 

A=l Ai ) i=l JA ' 



Now 



1 f w ~ 

Ga = — ; / D n (x + y)D n (— x) dx 



1 



2vr(ra+ 1) 

(A.5) 



x 



oo 



-oo 
G^ — Ge, 



sin(l/2)(z + (n + l)y) sin(l/2)z dz 
l\z\>(n+i)7r J (z + (n + l)y)/2(n + 1) z/2(n + 1) n + 1 

with G5 and Gg the two integrals of the previous line. Notice that 

|G 6 |<f{/ + / + / 

^ U(n+l)7r+l>|2|>(n+l)7r J2(n+l)7r>|z|>(n+l)7r+l J\z\>2(n+l)n 

sin(l/2)(z + (n + sin(l/2)z 



dz, 



(z + (n+l)y)/2 z/2 

with the integrand less than or equal to l/((n + 1)tt) in the first integral, 
less than or equal to ^ n ^ w (z + (n + in the second integral and less 

than 2/z 2 in the third integral (because \y\ < 7r). These bounds are enough 
to insure that Gq = 0(^^). In evaluating G5, we note that 

(a.6) s ":i::,' )!,> = r m-v^v*^™ *■ 



(z + (n + l)y)/2 

The Parseval relation implies that G5 = sinc(y). Applying the above bounds 
for the GiS proves the theorem. □ 

Proof of Theorem 2. By a direct calculation, we have four expres- 
sions of the same type as that analyzed in Theorem 1. Using (2.8) and (2.10), 
formula (2.11) is obtained. □ 

Proof of Corollary 1. The first equation follows directly from The- 
orem 2. The order of magnitude statement is obtained by making use of the 
Schwarz inequality. □ 
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Proof of Theorem 3. First notice that, for distinct indices in the 
range —n/2 to n/2, 

(A. 7) cum(X 2 , Xl , . . . , Xf ) = 2 s " 1 V r h ,• r,- ,• ... r,- ,-. , 

V 

where v is a listing of the (s — 1)1 irreducible sequences 1 — > oti — > ■ ■ ■ — > 
a s — ► 1 with ai2, ■ ■ ■ , oc s distinct choices of ji , . . . , j s . The case in which the 
indices are not distinct is obtained from (A. 7) by making the appropriate 
identification of identical indices. By (2.5), 

(A.8) \r juh \ <Y,Mi-ti)\- 

bee 

Since the q,'s are uniformly bounded by a summable c(-) > 0, 

(A.9) | r h jQ2 , r jaa , jaa . . . r jas dl \ < q s c(ji - j a2 )c{j a2 - j a3 ) • • • c(j as - 

We shall carry out an argument for Re{V n (b)} alone since that for Im{V n (w)} 
is similar. Let a n be the variance of Re{V n (w)} . Then 

logEexp[aRe{V n (w)}] - a 2 a n /2 

oo / oo \ 6 

<(n + l)£(2g)'cW £ /' 

s=3 \j=—oo / 

since the sth cumulant of V n (b) is less than or equal to (s— l)!(n+ l)(2q) s x 
(12 j C U)Y ■ If a > is sufficiently small, 

2 

2 



(A.ll) log£;e X p[aRe{K(6)}]<(n + l)g 2 ^c(i)^ (1 + e] 



a 



since 



0< ( T n = var(Re{KH}) 

/ n/2 x 

= var I X| cos (felt;) ) 

\k=-n/2 / 
n/2 

= 2 rj k cos(jw)cos(kw) 

j,k=-n/2 

<2(n + l)g 2 ^c(j)^ 

if {Xfc} is not the trivial process that is identically zero. By Lemma 2.1 and 
Corollary 2.1 of [19] (see also [21], Chapter 10, Theorem 7.28), 

(A.12) sup|Re{KHII < sup \Re{V n (uj)}\/(l - GnnR- 1 ), 
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with Uj = 2irj/R,j = 0, 1, . . . , R — 1 and R^> n. The simple argument for 
this follows from the fact that max^ |p'(A)| < nmax\ \p(X)\ for a trigonomet- 
ric polynomial p(X) = J2v=-n a v exp(zuA) of order n (Bernstein's inequality, 
page 11, [21], vol. 2). Then for Xj = 2irj/R,j = 0, 1, . . . , R - 1, 

2ir 

(A. 13) max |p(A)| < max |p(A,)| + — max |p'(A)|. 

|A|<7T \j\<R J R |A|<7T 

Equation (A. 13) implies that 



2n?r\ 
If) 



1 — max |p(A)| < max \p(Xj) 

|A|<* \j\<R' J/ 



and (A. 12) follows. Then 

-Eexp^ a sup | Re{Vn(w;)}| 



< Eexp 



(A.14) 



a supj | Re{ V n (uj ) } | 
(l-6irnR- 1 ) 

a\Re{V n ( Uj )}\ 



QirnR 



< 2exp jlogi? + [(i _ 6yrnjR „ 1)2] j 



Our objective is to show by an appropriate choice of a and R that 



(A.15) P 



suplRe-t^O)}! > 2a 



<Cn 



-1-5 



for some constant C. By (A.14) and Chebyshev's inequality, the probability 
on the left-hand side of (A.15) is less than or equal to 



exp{-2aa}2exp<J \ogR + (1 + e)(n + l)q 2 

(A.16) 



1 



E°0')) ^/[(l-GnnR- 1 )^. 



Given e, S > 0, set 



a 2 



(A.17) 

R = n log n 



(1 + e)(2 + 5)q 2 (n + 1) log(n + 1) ^ c(j)^ , 
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and 
(A.18) 



a = a(l — QnnR 



-1\2 



(l + e )(n + l)g 2 K>(i) 



On inserting these expressions for a, a and R, (A. 16) becomes 

(A.19) 2exp[-(l + <5)log(n + l)] < Cn -1-5 . 

The bound Cn _1_<5 has been obtained for sufficiently large n with an ap- 
propriate choice for the constant C . Thus, 



(A.20) 



P sup|Re{KH}|/(nlogn) 1 / 2 >2{(l + e)(2 + 5)} 1 / 2 ( ? Vc(j) 

L w V j 

By applying the Borel-Cantelli lemma to (A.20), one can see that, for rj > 0, 
(A.21) limsupsup | Ra{V n {w)}\ / {n\og n) l/2 < 2 3/2 < ? (l + 77) ( V c(J) ) . 

n— >oo uj y . J 

Since the same result holds for sup^, \ImV n (w)\, we obtain 
(A.22) lim sup sup | V n (w) \/{n log n) 1/2 < 2 5 / 2 (1 + rj)q I c{j) ] . 



□ 



Proof of COROLLARY 2. Notice that the probability of (A. 15) can 
be rewritten as P[sup w | Re{U n (w) — EU n (w)}\ > ^xf]- Let c > and a = 

cn/A, R = nlogn and a = n~ 1 / 2 (logn) 1 / 2 . Using (A. 16), we see that then 
there is a C > such that 



P 



sup\Re{U n {b) - EU n (b)}\ >- 



<exp(-Cn 1 / 2 (logn) 1 / 2 ). 



The same type of inequality holds for Im{U n (b) — EU n (b)}. Thus, the corol- 
lary holds. □ 

Proof of Corollary 3. The arguments are similar but now set a = 
en 1 / 2 (log n) 1 / 2 , R = nlogn and a = n~ 1 / 2 (logn) 1 / 2 with c>0 fixed but 
large. □ 



Proof of Corollary 4. Trivially, 



sup \U n {w) - EU n {w)\ 2 > 



1 

2^ 



U n (w) - EU n {w)\ 2 dw 
1 



n/2 

: £ (Xj-£X 2 ) 2 

i=-n/2 



n + 1 
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The covariance of ^(U n (w) - EU n {w)) and ^(U n (rj) - EU n (r])) is 



1 n/2 



£ cov(Xj,X fc 2 )e-^+^, 



(n+1) 

\ I j,k=-n/2 



with cov(Xj,X%) = 2r 2 - k since {Xt} is a Gaussian process. But then 
1 

(n+1) 5 



(1):=_L_ £ {^^(j-^)} 
j,k=-n/2 UeC J 



(n+1) 2 

w sin ((n + 1) /2)(b + b' - w + a) sin((n + 1) /2) (n - a) 



x V 

bvtJ^ sm(l/2){b + b' -w + a) sin(l/2)(n - a) 

x fb* fb'(oi)da, 

where "*" denotes convolution. If n = w — b — b' , we essentially get a Fejer 
kernel centered at this point and so 

( 1 ) = ^TT E ^-™ + & + &0/&*M") + °(4) 

a ^ 1 6,6' e£ v 7 

if there are a finite number of b in £. Notice that, for r/ = w, we get 

n + l^r^ \n z 

We first show joint asymptotic normality of U n {w) — EU n {w) for any finite 
number of w's. Also, there are an infinite number of Wi in a neighborhood 
of zero such that wi — Wj + b + b' / for any b, b' 6 £. For joint asymptotic 
normality, we have to deal with 

m 

(A.23) Y,[U n {Wj) - EUniwj)}^. 

J'=l 

The argument for estimates of the cumulants of V^(6) leading to Theorem 3 
still holds for the sth cumulant of (A.23) with the additional factor (J2 \ a j\Y ■ 
The asymptotic variance is 

( A - 24 ) ^n;Ew E 5k-^+6+&u*A'K)+o(^). 

b,b'£C 

We now consider conditions for positivity of the coefficient of 

We trivially get asymptotic normality for y/n^2jLi otj\U n {wj) — EU n {wj)\ 
but with a possibly singular distribution because the sth cumulants with 
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s > 2 all tend to zero. The asymptotic variance of U n {wj) — EU n {wj) is 
T^T Eft G £ fb * f-b(Wj) + 0(£). Now /o * / (0) = f (w) 2 dw>0if f is 
not identically zero. Further, for b £ C, 



fb * f-b(0) 



f b {w)f_ b (-w)dw 



\fbH\ 2 dw>0. 



Thus, J2b£C h * f-b(0) > and this implies J2bec fb * f~b(v) > C > for rj in 
some neighborhood of zero. But there are a countable number of Wi ^ in 
this neighborhood such that Wi — Wj + b — b' ^ for any b, b' £ C. Therefore, 
the \fn\U n (wj) — EU n (wj)} are asymptotically jointly normal and indepen- 
dent with variance greater than 5 > 0. Therefore, 



1 = 



SUp \y/n[U n (Wi) - EU n (Wi)}\ 



sup \ V^[U n (w) - EU n {w)}\ 



in probability. We have the corollary. □ 



Proof of Corollary 5. The proof is similar to that of Theorem 4 
and is omitted. □ 

Proof of Theorem 5. If A = /x + w, Theorem 1 gives 

Ei n (\,»)= £ A(,) sin 5 ( " + 1 ?/ 2)# ' w)) + o /logB 



bee 

\y(b,w)\<ir 



Since 



it follows that 



fw n (v) 



Ef Wn{b) (v)=Y, E 
b'ec 



((n+l)/2)y(b,w) 

I n ((j, + w n , /J.)K n (p - rj) du, 
sm((n + l)/2)y(b',w n (b)) 



;? 



E 



((n + l)/2)y(&>„(6)) 

/ n ( log n 

fb>{n)K n (n-ri) d/i + O' 

sin((n + l)/2)y(6,it> n (6)) 



((n + l)/2)y(b,w n (b)) 



(l) + (2), 



where 

(1) :=E 

(2) :=E 



sm({n + l)/2)y(b,w n (b)) 
((n + l)/2)y(6,w n (6)) 

sm((n + l)/2)y(b,w n (b)) 
((n + l)/2)y(b,w n (b)) 



;\n(b-w n (b))\ < kn~ l / 4 (log n) 1 ' ' 4 
; \n(b - w n (b))\ > kn~ l l A {\ogn) l / A 
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with E[f(X);A] := J A f(x)dP{x) where P(-) is the distribution of X. Now 

(l)=E[l-[n(b-w n (b))} 2 /3\ 

+ o([n(b - w n (b))] 2 ); \n(b - w n (b))\ < fen" 1 / 4 (logn) 1 / 4 ]. 

By Corollary 3, 1 > (1) > (1 - A: 2 n^ 1 / 2 (logn) 1 / 2 )(l - ±), while |(2)| < ± if k 
is large enough. Therefore, 

-sm((n + l)/2)y(b,w n (b)Y 



E 



((n+l)/2)y(6,w n (6)) 

Now if b' 6 consider 

- sin((n + l)/2)^(6 / , Wn (6)) - 
. ((n + l)/2)y(b',w n (b)) . 

sin((n + l)/2)i/(&>„(&)) 



^^n-V^bgn) 1 / 2 . 



where 



E 

(I') :=£ 



((n + l)/2)y(6',^ n (&)) 
sin((n + l)/2) 2 /(6 / ,w n (6)) 



= (!') + (2'), 

;|n(6-w n (6))| >c 
;|n(6- w n (6))| < c 



((n + l)/2)y(&>„(&)) 

By Corollary 2 |(l')| < exp(-Cn 1 / 2 (log?i) 1 / 2 ) + 0(n~ 2 ),\(2')\ <C/n. No- 
tice that |( H y^)(6 — u> n (&))| 2 < fcsup \U n (w) — EU n (w)\ except for a set of 
probability 0(n~ 2 ), which implies 

P[\(n + l)(b-w n (b))\ < en" 1 / 2 (log n) 1 / 2 } > 1 - exp[-c' logn] 

for some d and so P[\ (n + 1)(6 — w n (b))\ > cn~ 1 / 2 (logn) 1 / 2 ] < exp[— d logn]. 
These estimates imply that, if f b is twice continuously differentiate, Ef Wn ^ (rj) 
fb(v) + §fl'(v) I x 2 K(x) dx + o(b 2 n ) + O^-V^logn) 1 ^). D 

Proof of Theorem 6. First notice that 
cov(f Wri{b) (r)), f Wn{v) (r]')) 

= E [ E [fw n (b)(v)fw„(b')(v')\w n (b) =w n ,w n (b') =w' n ]] 

- E fw n (b)(v) E fw n (b>)(v) 

E[cOv(f Wn (b)(v)Jw n (b')('n')\ w n(b) =W n ,W n (b') = w' n )] 

+ E[E(f Wn{b) (r])\w n {b) = w ri ,w n (b , ) = w' n ) 

x E {fw n (b')(v')\w n (b) = w n ,w n (b') = w' n )} 

- E fw n {b)(v) E fw n (b')(v')- 



(A.25) 
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The estimate for the first term on the right-hand side of (A. 25) is obtained 
by making use of Corollary 1. The remaining terms on the right-hand side 
of (A. 25) can be seen to be 

-sm((n + l)/2)y(b",w n (b)) sin((n + l)/2)y(b"', w n (b')Y 



b",b"'eC 



((n + l)/2)y(b",w n (b)) ' ((n + l)/2)y(b"',w n (b')) 
x r f b {ii)K n {n-ri)dli [* fvi^K^n-^dn + of 1 ^). 

J-TT J-TT \ U J 

Of course, 

| cov (sine y (b" , w n (6) ) , sine y {b'" ,w n (b')))\ 

< { var (sine y {b" ,w n (b))) var (sine y{b"' , w n (b') ) ) } 1/2 . 

Notice that var(sincy(6 / , w n (b))) is 

E[(smcy(b",w n (b))) 2 } - (E[smcy(b" ,w n (b))}) 2 

= E[(l + {( S mcy(b",w n (b))) - l}) 2 ] - [E(l + {sincy(&", w n {b)) - l})] 2 

= E[{smcy(b",w n (b)) - l} 2 ] - (E[{ S mcy(b", w n (b)) - l}]) 2 . 

One has to consider the case in which b = b" and that in which b ^ b" . 
Also, one should note that the expression above is less than or equal to 
E[{smcy(b",w n (b)) - l} 2 ]. If 6 = 6", 

E[{smcy(b,w n (b)) - l} 2 ; |n(6 - w n (b))\ < fcn-^^logn) 1 / 4 ] 

EU f^n + l)/2){b-w n {b))\ 2 , J((n + l)/2)(b-w n (b))^ 



"3! -) ■"{- — 

\n(b - w n (b))\ < kn-^^logn) 1 ^ 

< Clogn/n, 
while, by Corollaries 3 and 4, 

£[{sincy(6,u; n (6)) - l} 2 ; \n(b-w n {b))\ > fcn" 1/4 (logn) 1/4 ] < c/n 

if k is large enough. If 6 ^ 6", 

£■[{ sine 1/(6", w n (b)) - l} 2 ; \n(b - w n (b))\ < c] < c/n 

because w n (b) is close to 6 and b" — w n {b) is large, while 

E[{ sine 1/(6", w n (b)) - l} 2 ; \n(b - w n (b))\ > c] 
< e -CnV 2(logn) i/ 2+0(n _ 2) 

follows by Corollary 2 as n — > oo. The proof of Theorem 6 is complete. □ 
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Proof of Theorem 7. 

var(/ u , n(6) (r?)) = E(vai(f Wnib) (i])\w n (b) = w n )) 
(A.26) +E[E(f Wn{b) (r ] )\w n (b) = w n ) 

-{Ef Wn{b) (ri)f. 
We first consider the first term on the right-hand side: 
va*(fw n (b)(v)\wn(b) =w n ) 

'logn 



£/ 6 (// + <)sinc(y(l)) + C> 
x jE/f'(V)sinc(y(2)) + 
+ |E^(V)sinc(y(3)) + 



logn 



logn 



n 



log 77, 



77 



x \ S A'(M / + ^n)sinc(y(4)) + 

x K n (/j - r])K n (fi' - t?) cfyid//. 

Making use of the symmetry of .fT about zero on taking the expectation, we 
obtain 

var(f Wn {b)(v)\w n {b)=w n ) 

2tt 
nb n 



Mv + b)M-v) 



+ J2rt-2r ) + b , -b)\f b/ (- V )f 

b'&C 



/oo 
K 2 (x)dx + 
-oo 



logn 



77 



The term 



E[E(f Wn(b) (ri)\w n (b) = w n )} 2 - {Ef Wn{b) ( V )} 2 



can be shown to be O(^p) just as in the discussion of the corresponding 
term in the proof of Theorem 6. □ 
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